Combinatorial Q-Learning for Condition-Based Infrastructure Maintenance

Infrastructure maintenance planning is a large-scale optimization problem of when and on which components to carry out so as keep the whole infrastructure in good condition with minimal cost. Recent advances monitoring techniques have enabled timely response each part regardless age. In addition condition, spatial structure also important for cost-efficiency since traveling costs and/or setup can be saved by simultaneous neighboring components, called economic dependency. This naively has high computational complexity $O(2^{nH})$ , where notation="LaTeX">$n$ number notation="LaTeX">$H$ horizon, predictive modeling degradation big issue. To solve this efficiently at scale, our proposed method utilizes two kinds dynamic programming temporal scalability consequently enjoys notation="LaTeX">$O(n)$ time step. For scalability, we utilize direct approach action value instead degradation, namely, Q-learning. exploit locality dependency means reasonable approximation Q-function. A typical baseline divide into fixed groups beforehand determine if should performed all group contrast, scalable enables fully combinatorial component We demonstrate advantage simulated environment, resulting history intuitively illustrates benefit grouping approach. show that kind interpretability